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Bovine respiratory syncytial virus (BRSV) and bovine coronavirus (BCV) are responsible for respiratory disease 
and diarrhea in cattle worldwide. The Norwegian control program against these infections is based on herd-level 
diagnosis using a new multiplex immunoassay. The objective of this study was to estimate sensitivity and 
specificity across different cut-off values for the MVD-Enferplex BCV/BRSV multiplex, by comparing them to a 
commercially available ELISA, the SVANOVIR* BCV-Ab and SVANOVIR* BRSV-Ab, respectively. We analyzed 
bulk tank milk samples from 360 herds in a low- and 360 herds in a high-prevalence area. As none of the tests 
were considered perfect, estimation of test characteristics was performed using Bayesian latent class models. At 
the manufacturers’ recommended cut-off values, the median sensitivity for the BRSV multiplex and the BRSV 
ELISA was 94.4 [89.8-98.7 95% Posterior Credibility Interval (PCI)] and 99.8 [98.7-100 95% PCI], respectively. 
The median specificity for the BRSV multiplex was 90.6 [85.5-94.4 95% PCI], but only 57.4 [50.5-64.4 95% 
PCI] for the BRSV ELISA. However, increasing the cut-off of the BRSV ELISA increased specificity without 
compromising sensitivity. For the BCV multiplex we found that by using only one of the three antigens included 
in the test, the specificity increased, without concurrent loss in sensitivity. At the recommended cut-off this 
resulted in a sensitivity of 99.9 [99.3-100 95% PCI] and specificity of 93.7 [88.8-97.8 95% PCI] for the mul¬ 
tiplex and a sensitivity of 99.5 [98.1-100 95% PCI] and a specificity of 99.6 [97.6-100 95% PCI] for the BCV 
ELISA. 


1. Introduction 

Bovine coronavirus (BCV) and bovine respiratory syncytial virus 
(BRSV) are commonly occurring agents among cattle worldwide 
(Valarcher and Taylor, 2007; Boileau and Kapil, 2010). They are en¬ 
demic and prevalent also in the Norwegian dairy herd (Gulliksen et al., 
2009; Klem et al., 2014a). BCV causes respiratory disease, calf diarrhea 
and winter dysentery (contagious diarrhea in adult cattle) (Boileau and 
Kapil, 2010). BRSV causes respiratory disease mostly in young animals 
but can affect animals of all ages, and is a common cause of respiratory 
outbreaks in Norway (Larsen, 2000; Klem et al., 2014a). Consequences 
of these infections are herd health problems, reduced animal welfare 
and increased use of antibiotics due to secondary bacterial infections 


(Larsen, 2000; Valarcher and Taylor, 2007; Boileau and Kapil, 2010). 

Therapy costs and reduced production entails considerable financial 
loss for the farmer, and contributes to a present focus in Nordic coun¬ 
tries on how to limit the spread of these viruses in the cattle population. 

In 2016, a national control program against BRSV and BCV infec¬ 
tions was launched in Norway as a joint initiative between the producer 
organizations. This prompted the need for an easy and cost-effective 
way to screen dairy herds for a herd level diagnosis for BRSV and BCV. 
The initial screening in the control program was conducted using bulk 
tank milk samples (BTM). There are already commercially available 
indirect enzyme-linked immunosorbent assays (ELISAs) widely used in 
routine diagnostics and research in the Nordic countries (SVANOVIR* 
BRSV-Ab and SVANOVIR* BCV-Ab) (Traven et al., 1999; Klem et al., 
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2014b; Toftaker et al., 2016). However, in order to optimize cost-ef¬ 
fectiveness of the control program, the development of a new multiplex 
antibody ELISA was initiated (MVD-Enferplex BCV/BRSV multiplex). 
The new test allowed screening for both viruses by the use of a single 
test. 

The performance of a diagnostic test is characterized by the test’s 
sensitivity (Se) and specificity (Sp), where Se is the proportion of true 
positives correctly classified as positive by the test, and the Sp is the 
proportion of true negative subjects correctly classified as negative. The 
true antibody status of each test subject can be determined in two ways: 
By use of a perfect reference test, or based on populations with known 
status. However, a perfect reference test (often termed a “gold stan¬ 
dard”) is rarely available and for endemic diseases, which is the case for 
BRSV and BCV in Norway, no reference population with complete 
certainty regarding disease or disease freedom exists. Consequently, the 
underlying true infection status for test subjects remains unknown. Test 
validation studies (erroneously) assuming perfect reference tests are 
common, even though this has been shown to introduce bias in the 
estimation of accuracy parameters (Valenstein, 1990; Lijmer et al., 
1999). Latent class analysis (LCA) allows for the estimation of test 
parameters in populations where the underlying true infection status 
cannot be determined (Hui and Walter, 1980). In LCA the true infection 
status is treated as an existing, but unknown (latent), variable and test 
accuracy and prevalence are parameterized according to this latent 
variable. 

As the BRSV/BCV multiplex is a new test, it needs to be validated. 
Test characteristics are different when a test is used as a herd test, 
compared to when it is used on individual samples (Christensen and 
Gardner, 2000) and validation for the relevant application is therefore 
important. BTM testing is a key component of the Norwegian BRSV/ 
BCV control-program, it is therefore of interest to estimate test accu¬ 
racy, at different cut-off values, for this application. 

The aim of this study was to estimate the test sensitivity and spe¬ 
cificity of the newly developed MVD-Enferplex BCV/BRSV multiplex 
across different cut-off values, for detection of antibodies in BTM. The 
BCV part of the multiplex was compared to the commercially available 
SVANOVIR BCV-Ab, and the BRSV part of the multiplex was compared 
to the SVANOVIR* BRSV-Ab. As neither test could be considered per¬ 
fect, the evaluation was done using LCA. 

2. Materials and methods 

2.1. Study population and sample material 

A cross-sectional sampling design was used for the present study. 
Herds were eligible for inclusion if they delivered milk to the largest 
dairy company in Norway (TINE SA), and provided a BTM sample 
during the study period (March 2016). Herds from two counties with an 
expected difference in true prevalence (TP) were selected in order to 
meet the model assumptions, described in the LCA section. Using a 
random numbers generator, 360 samples were randomly chosen from 
herds in “Oppland” (Pop 1) and 360 from herds in “Sogn og Fjordane” 
(Pop 2) counties. “Sogn og Fjordane” is located in western Norway, and 
was assumed to have a relatively low prevalence, based on results from 
a previous study (Toftaker et al., 2016). Oppland county, located in 
eastern Norway, was thought to have higher prevalence based on 
known patterns of animal movements and a history of previous out¬ 
breaks of disease (Toftaker et al., 2017). 

BTM samples were collected from nearly all Norwegian dairy herds 
delivering milk to the largest dairy cooperation (TINE SA) during March 
2016. The samples were collected as part of the national control pro¬ 
gram against BRSV and BCV. The milk truck driver collected samples at 
ordinary milk shipment using standard procedures for BTM sampling. 
The milk was then stored at 4 °C until received at the laboratory (TINE 
Mastitis Laboratory, Molde, Norway) where samples were frozen and 
shipped over-night to the Enfer laboratory in Ireland (Enfer Scientific, 


Naas, Ireland). Samples were kept frozen until the time of laboratory 
analysis. 

2.2. Diagnostic tests 

2.2.1. ELISA 

The SVANOVIR* BRSV-Ab, hereafter designated the BRSV ELISA, 
and SVANOVIR* BCV-Ab, hereafter designated the BCV ELISA, were 
used on all 720 samples, following the manufacturer’s instructions. The 
optical density (OD) reading of 450 nm was corrected by the subtrac¬ 
tion of OD for the negative control antigen, and percent positivity (PP- 
value) was calculated as (corrected OD/positive control corrected 
OD) x 100. According to the test manuals, the recommended cut-off 
values of sample positive > 10 PP for both tests were used as a starting 
point for these tests (Svanova; Svanova). For the BRSV ELISA the Se and 
Sp provided by the manufacturer were 94% and 100%, respectively. 
These parameters are calculated from serum samples, and parameters 
specific for BTM samples have not been reported (Elvander et al., 
1995). For the BCV ELISA the test parameters provided by the manu¬ 
facturer were Se of 84.6% and Sp of 100%, and as for BRSV the cal¬ 
culations are based on serum samples (Alenius et al., 1991). 

2.2.2. Multiplex 

All 720 samples were analyzed using the MVD-Enferplex BCV/BRSV 
multiplex, hereafter referred to as the BRSV/BCV multiplex (Enfer 
Scientific, Naas, Ireland). A panel of three BCV recombinant proteins 
(BCV A-C), along with a panel of two recombinant proteins and two 
synthetic peptides for BRSV (BRSV A-D) were used as antigens. Briefly, 
the antigens were deposited in a multiplex planar array as individual 
spots into wells of 96 well microtiter plates to produce arrays of anti¬ 
gens. Samples were diluted 1:3 into sample dilution buffer and mixed 
before added to the well and incubated at 37 °C for 60 min with agi¬ 
tation. After washing procedures, the detection antibody diluted in 
conjugate buffer was added and plates were incubated (37 °C for 60 min 
with agitation) before new washing. Finally, the chemiluminescent 
substrate was added. Relative light units (RLU) were captured (45 s 
exposure) immediately, using Quansys biosciences imaging system, and 
data was extracted using Quansys Q view software (v 1.5.4.7). Antigens 
were combined in a parallel reading, i.e. the test was considered posi¬ 
tive when the RLU-value of at least one antigen was above the applied 
cut-off. Laboratory personnel were not formally blinded to test results, 
but due to the large volume of samples they were considered blinded 
for any practical purposes. 

2.3. Data management and descriptive statistics 

Because the multiplex consisted of several antigens each giving a 
separate response, a separate cut-off value was needed for each antigen. 
We calculated the proportion of herds that had a positive response to 
each of the individual antigens within the test-positive group (at 
manufacturers recommended cut-off values), and defined the antigen 
with the highest proportion of positive responses as the most influen¬ 
tial. This was done for both viruses. When later choosing which cut-off 
values to assess, changing the cut-off for the most influential antigen for 
each virus was prioritized. We used an explorative approach to se¬ 
lecting cut-off values, and several different cut-off values were tried for 
the most influential antigen (Fig. 1). Furthermore, we evaluated test 
performance when including only the single most important antigen. 
Data preparation and descriptive analysis were performed in Stata 
(Stata SE/14; Stata Corp., College Station, TX). 

2.4. Latent class analysis 

In the present study, we used guidelines for reporting of diagnostic 
accuracy in studies that use Bayesian LCA (Kostoulas et al., 2017). 

The target condition was herds with one or more animals producing 
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Cut-off 

alternative 

BRSV-A 

BRSV -B 

BRSV-C BRSV-D 

1 

1000 

4000 

7000 

1700 

2 

2000 

4000 

7000 

1700 

3 

3000 

4000 

7000 

1700 

4 

4000 

4000 

7000 

1700 

5 

4000 

6000 

9000 

2000 

6 

SOOO 

4000 

7000 

1700 

7 

2000 

- 

- 

- 

S 

4000 

- 

- 

- 

9 

2000 

4000 

7000 

1700 


Cut-off 

alternative 

BCV-A 

BCV-B 

BCV-C 

1 

10000 

11000 

30000 

2 

15000 

15000 

35000 

3 

20000 

15000 

35000 

4 

25000 

15000 

35000 

5 

30000 

15000 

35000 

6 

35000 

15000 

35000 

7 

40000 

15000 

35000 

S 

10000 

- 

- 

9 

35000 

- 

- 


1 







Fig. 1. To the left are the different cut-off values 
(relative light units) for the BRSV antigens (top) and 
BCV antigens (bottom) included in the BCV/BRSV 
multiplex. To the right are spider plots of median Se 
and Sp for the different cut-off alternatives. The 
BRSV ELISA cut-off was fixed at sample positive > 
10 PP, except for alternative 9 where sample posi¬ 
tive > 50 PP was used. For the BCV ELISA the cut-off 
was fixed at sample positive >10 PP. Test para¬ 
meters are estimated from a Bayesian LCM analysis. 



BCV/BRSV-antibodies while contributing to the bulk tank. The under¬ 
lying latent state could be considered as previous exposure, leading to 
antibodies in BTM. 

The use of LCA methodology for diagnostic test evaluation requires 
a set of assumptions of the tests and test populations to be fulfilled. (1) 
two or more populations with different prevalence are included, (2) the 
Se and Sp of the diagnostic tests are the same across the populations, 
and (3) the tests are conditionally independent (CID) given disease 
status (Hui and Walter, 1980). We ran the analyses assuming CID be¬ 
tween tests; however, we also explored the consequences of relaxing 
this assumption as explained below. For the CID-models, parameters 
were estimated for several cut-off values (Fig. 1). Models were fit using 
Bayesian LCA in the OpenBugs version 3.2.1 rev 781 software. We used 
non-informative priors in the shape of uniform distributions on the 
interval between zero and one, modelled using the beta (1, 1) dis¬ 
tribution for test properties and sub-population prevalence in all ana¬ 
lyses. Models were run with 20,000 iterations, of which 10,000 were 
used as burn in and discarded. Convergence of the Markov Chain Monte 
Carlo (MCMC) chains were assessed by visual inspection of history 
plots, time-series plots and Gelman-Rubin diagnostic plots using three 
sample chains with different initial values, as suggested by Toft et al. 
(2007). Posterior inference was done by calculating medians and 95% 
posterior credibility intervals (PCI) for Se, Sp and true prevalence. The 
model description is included in Appendix A. 

2.5. Sensitivity analysis 

A correlation between tests, if present, is not possible to estimate in 
a two tests scenario without including informative priors. We did not 
have any reliable prior information on test performance or prevalences 
in the present study. However, the consequences of relaxing the as¬ 
sumption of conditional independence given disease status was first 
explored by Vacek (1985), who examined the impact of conditional 
dependence by assuming a fixed proportion of the maximum possible 
covariance between tests. Following this approach we explored the 
consequences of conditional dependence between tests for the cut-off 
values with the preferred test characteristics. (Fig. 1: alternative 2 for 
BRSV, alternative 8 for BCV.) See Appendix A for details. 

We compared the results of the conditional independence model to 


models allowing 25, 50, 75 and 90% of the maximum possible positive 
covariance, as well as a negative covariance of - 25%. 

3. Results 

3.1. Descriptivestatisti.es 

A combination of different cut-off values for the included antigens, 
(cut-off alternatives 1-9) are presented in Fig. 1 for the BRSV- and BCV 
multiplex. For the BRSV multiplex, the BRSV-A antigen was responsible 
for detecting the majority of the positive samples. For the BCV multi¬ 
plex the antigen detecting the majority of positive samples was the 
BCV-A. Counts of test outcomes for the tests are presented in Tables 1 
and 2 for the BRSV and BCV tests, respectively. 


Table 1 

Counts of paired test outcomes in the two sub-populations for the BRSV-antibody tests 
(BRSV multiplex/BRSV ELISA). For the BRSV multiplex varying cut-off values for the 
included antigens were used (shown in Fig. 1). The BRSV ELISA cut-off was fixed at 
sample positive >10 PP, except for alternative 9 where sample positive > 50 PP was 
used. 


BRSV multiplex/BRSV ELISA 


Cut-off alternative 

Pop 1 




Pop 2 




+ / + 

+ /- 

-/+ 

-/- 

+ / + 

+ /- 

-/ + 

-/- 

1 

299 

0 

35 

26 

111 

16 

103 

130 

2 

299 

0 

35 

26 

111 

16 

103 

130 

3 

287 

0 

47 

26 

105 

16 

109 

130 

4 

283 

0 

51 

26 

102 

16 

112 

130 

5 

272 

0 

62 

26 

94 

12 

120 

134 

6 

274 

0 

60 

26 

93 

15 

121 

131 

7 

289 

0 

45 

26 

107 

12 

107 

134 

8 

264 

0 

70 

26 

84 

11 

130 

135 

9 a 

295 

4 

12 

49 

105 

22 

18 

215 


a BRSV ELISA cut-off: sample positive > 50 PP. 


3 















I. Toftaker et aL 


Preventive Veterinary Medicine 154 (2018) 1-8 


Table 2 

Counts of paired test outcomes in the two sub-populations for the BCV-antibody tests 
(BCV multiplex/BCV ELISA). For the BCV multiplex varying cut-off values for the in¬ 
cluded antigens were used (shown in Fig. 1). The BCV ELISA cut-off was fixed at sample 
positive >10 PP. 


BCV multiplex/BCV ELISA 


Pop 1 Pop 2 


Cut-off alternative 

+ / + 

+ /- 

-/ + 

-/- 

+ / + 

+ /- 

-/ + 

-/- 

1 

336 

7 

0 

17 

219 

34 

0 

107 

2 

335 

3 

1 

21 

215 

14 

4 

127 

3 

334 

2 

2 

22 

207 

11 

12 

130 

4 

330 

2 

6 

22 

198 

9 

21 

132 

5 

329 

2 

7 

22 

187 

9 

32 

132 

6 

324 

2 

12 

22 

182 

9 

37 

132 

7 

301 

2 

35 

22 

174 

9 

45 

132 

8 

336 

2 

0 

22 

219 

10 

0 

131 

9 

321 

0 

15 

24 

180 

1 

39 

140 


3.2. Latent class analysis 
3.2.1. BRSV 

Estimates of median Se and Sp and true prevalence in the two sub¬ 
populations for the BRSV-multiplex and BRSV ELISA when applying 
different cut-off values are presented in Table 3. As a starting point the 
recommended cut-off values from the test manufacturers were applied 
(alternative 2 in Fig. 1), resulting in median Se of 94.4 and Sp of 90.6 
for the BRSV multiplex, and Se 99.8 and Sp 57.4 for the ELISA. The Sp 
of the ELISA increased to 99.4 (Se 93.4) when a cut-off of sample po¬ 
sitive > 50 PP was used. For the multiplex, increasing the cut-off value 
for the BRSV-A antigen generally resulted in lower Se and higher Sp 
estimates as could be expected. Discarding all antigens except the 
BRSV-A resulted in increased specificity, however, with the cost of 
significantly reduced sensitivity, as can be seen from comparing cut-off 
alternative 2 and 7 in Table 3. Point estimates (median) of true BRSV 
antibody prevalence ranged from 84.5 to 87.3 for pop 1, and from 25.2 
to 30.5 for pop 2. Results from the COC-models with fixed covariance, 
showed that allowing for covariance altered specificity estimates for 
both the ELISA and the multiplex. The change was small for a covar¬ 
iance of 0.25 or less of the maximum possible covariance. The Se es¬ 
timates were not noticeably affected by allowing for covariance. Results 
from the sensitivity analysis are presented in Table 4. 


3.2.2. BCV 

Estimates of test parameters and true prevalence in the two sub¬ 
populations across different cut-off values for the BCV multiplex and 
the BCV ELISA are presented in Table 5. When we applied the cut-off 
values currently recommended by the test manufacturers (alternative 1 
in Fig. 1), the estimated median Se and Sp was 99.9 [99.4-100 95% 
PCI] and 77.3 [69.8-84.8 95% PCI], for the BCV multiplex, and 99.0 
[96.9-100 95% PCI] and 99.5 [97.1-100 95% PCI] for the BCV ELISA, 
respectively. Similar to what we observed for BRSV, increasing the cut¬ 
off for the most important antigen (BCV-A) resulted in a lower Se and a 
higher Sp for the BCV multiplex. When we used the BCV-A as the sole 
antigen (cut-off alternative 8, Table 5) the median Sp increased to 93.7 
while the median Se remained unchanged (99.9). Point estimates 
(median) of true BCV antibody prevalence ranged from 91.5 to 94.0 for 
pop 1, and from 52.4 to 61.5 for pop 2. Results from the sensitivity 
analysis, i.e. allowing for covariance between tests, showed negligible 
effect on the estimated test-parameters; less than 5% change in para¬ 
meters for covariance at 75% of maximum possible (results not shown). 

4. Discussion 

We estimated the sensitivity and specificity of a new multiplex and 
two commercial ELISAs for detection of BRSV and BCV antibodies in 
BTM using LCA. This is the first study evaluating the MVD-Enferplex 
BRSV/BCV multiplex. The present study is also the first to present test 
parameters for the SVANOVIR* BRSV-Ab and SVANOVIR BCV-Ab on 
BTM. The BRSV multiplex showed a somewhat lower Se, but a much 
higher Sp than the BRSV ELISA at the recommended cut-off values. 
However, when we increased the cut-off of the BRSV ELISA to sample 
positive > 50 PP, this resulted in a large increase in Sp without a no¬ 
table decrease in Se, as shown in Table 3. Our results therefore suggest 
that a higher cut-off than recommended by the manufacturer might be 
appropriate when using the SVANOVIR BRSV-Ab on BTM. For BCV, 
the specificity of the multiplex was notably lower than the BCV ELISA at 
the recommended cut-off when using all three antigens. However, when 
using the BCV-A antigen only, the Sp improved without the cost of 
reduced Se, and the test performance was then similar to the BCV 
ELISA. This implies that the extra antigens are adding false positive 
samples, hence reducing Sp. Overall; the two tests in this study both 
showed good performance for detection of both BRSV and BCV anti¬ 
bodies. A possible benefit of choosing the multiplex therefore lies in 
enabling screening for both agents simultaneously as this will reduce 
screening costs. As the multiplex evaluated in the present study is a new 


Table 3 

Test parameter estimates for the BRSV multiplex and BRSV ELISA: Sensitivity, specificity, and estimates of true prevalence (TP) in the two sub-populations. Cut-off alternative 1- 9 
represents different cut-off alternatives for the BRSV multiplex (presented in Table 1). The BRSV ELISA cut-off was fixed at sample positive > 10 PP for all alternatives except for 
alternative 9, where the BRSV ELISA cut-off was increased to sample positive > 50 PP. 


Test Sub-population 



BRSV multiplex 



BRSV ELISA 



Pop 1 


Pop 2 


Parameter 

Se 


Sp 


Se 


Sp 


TP 


TP 


Cut-off 

alternative 

Median 

[95% PCI] 

Median 

[95% PCI] 

Median 

[95% PCI] 

Median 

[95% PCI] 

Median 

[95% PCI] 

Median 

[95% PCI] 

1 

94.4 

[89.8;98.7] 

90.6 

[85.5;94.4] 

99.8 

[98.7;100] 

57.4 

[50.5;64.4] 

87.2 

[81.7;91.5] 

29.9 

[24.1;35.9] 

2 

94.4 

[89.8;98.7] 

90.6 

[85.5;94.4] 

99.8 

[98.7;100] 

57.4 

[50.5;64.4] 

87.2 

[81.7;91.5] 

29.9 

[24.1;35.9] 

3 

90.7 

[85.6;96.0] 

90.6 

[85.6;94.4] 

99.7 

[98.6;100] 

56.7 

[49.7;63.9] 

86.8 

[83.5;92.8] 

29.2 

[25.6;39.4] 

4 

89.5 

[84.3;95.0] 

90.5 

[85.5;94.4] 

99.7 

[98.6;100] 

56.3 

[49.2;63.4] 

87.0 

[80.8;91.4] 

28.5 

[22.5;34.9] 

5 

86.3 

[80.8;92.0] 

92.9 

[88.3;96.2] 

99.7 

[98.5;100] 

55.8 

[48.8;62.9] 

86.8 

[80.6;91.4] 

27.8 

[21.8;34.4] 

6 

87.0 

[80.3;91.2] 

91.1 

[86.3;94.8] 

99.7 

[98.6;100] 

54.8 

[47.7;62.0] 

86.6 

[80.3;91.2] 

26.5 

[20.4;33.0] 

7 a 

91.2 

[86.3;96.3] 

92.9 

[88.3;96.2] 

99.7 

[98.6;100] 

57.9 

[50.8;65.1] 

87.3 

[81.4;91.7] 

30.5 

[24.6;36.7] 

8 a 

84.1 

[78.4;90.2] 

93.4 

[89.1;96.6] 

99.7 

[98.5;100] 

53.9 

[47.1;60.8] 

86.4 

[80.0;91.1] 

25.2 

[19.3;31.7] 

9 b 

97.0 

[94.0;99.2] 

91.5 

[87.6;94.6] 

99.4 

[97.5;100] 

93.4 

[89.0;97.0] 

84.5 

[80.1;88.2] 

30.1 

[25.1;35.4] 


Only BRSV-A antigen included. 

BRSV ELISA cut-off: sample positive > 50 PP. 


4 
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Table 4 

Results from the sensitivity analysis (BRSV): Median estimates and 95% posterior credibility intervals (PCI) of the sensitivity (Se) and specificity (Sp) of bulk tank milk BRSV multiplex 
and BRSV ELISA at the manufacturers’ recommended cut-off (alternative 2, Fig. 1), for the conditionally independent (CID) model and conditionally dependent (COC) models where the 


covariance is expressed as 

proportions of maximum 

possible value. 







Conditional covariance 3 

BRSV multiplex 




BRSV ELISA 





Se 


Sp 


Se 


Sp 



Median 

[95% PCI] 

Median 

[95% PCI] 

Median 

[95% PCI] 

Median 

[95% PCI] 

CID model 

0.00 

94.4 

[89.8;98.7] 

90.6 

[85.5;94.4] 

99.8 

[98.7;100] 

57.4 

[50.5;64.4] 

COCseandSp 

0.25 

94.1 

[89.5;98.6] 

87.3 

[80.5;92.5] 

99.6 

[98.1;100] 

55.2 

[48.3;62.3] 

0.50 

93.6 

[88.8;98.3] 

80.7 

[70.6;88.6] 

99.4 

[96.9;100] 

50.7 

[43.3;58.1] 

0.75 

92.1 

[84.9;97.6] 

69.3 

[62.5;79.4] 

98.5 

[93.2;100] 

42.7 

[36.8;50.3] 

0.9 

89.8 

[82.1;96.6] 

67.8 

[62.1;75.4] 

97.1 

[91.7;99.9] 

40.6 

[35.2;46.8] 

-0.5 

94.5 

[89.9;99.0] 

93.7 

[90.4;96.3] 

99.8 

[99.1;100] 

59.6 

[52.5;66.9] 

COCse 

0.25 

94.1 

[89,5;98.6] 

90.5 

[85.6;94.4] 

99.7 

[98.1;100] 

57.4 

[50.5;64,4] 

0.50 

93.6 

[88,7;98.4] 

90.5 

[85.5;94.3] 

99.4 

[96.9;100] 

57.4 

[50.5;64.6] 

0.75 

91.8 

[84.3;97.5] 

90.2 

[84.8;94.2] 

98.3 

[92.8;100] 

57.3 

[50.2;64.4] 

0.9 

87.4 

[81.1;95.6] 

89.4 

[83.5;93.7] 

95.4 

[90.9;99.7] 

57.0 

[49.9;64.1] 

-0.25 

94.3 

[89.8;98.7] 

90.5 

[85.5;94.4] 

99.8 

[98.7;100] 

57.4 

[50.5;64.5] 

COCsp 

0.25 

94.3 

[89.7;98.8] 

87.3 

[80.6;92.5] 

99.8 

[98.7;100] 

55.2 

[48.3;62.3] 

0.50 

94.3 

[89.8;98.7] 

80.9 

[70.6;88.6] 

99.8 

[98.6;100] 

50.8 

[43.3;58.3] 

0.75 

94.3 

[89.7;98.8] 

69.6 

[62.7;79.5] 

99.7 

[98.6;100] 

43.0 

[37.0;50.4] 

0.9 

94.3 

[89.6;98.7] 

68.2 

[62.3;77.0] 

99.7 

[98.3;100] 

40.8 

[35.4;47.7] 

-0.25 

94.4 

[89.7;98.8] 

92.0 

[87.8;95.3] 

99.7 

[98.7;100] 

58.4 

[51.7;65.3] 


a Proportion of upper limit of conditional covariance. 


Table 5 

Test parameters for the BCV multiplex and BCV ELISA: Sensitivity, specificity, and estimates of true prevalence (TP) in the two sub-populations. Cut-off alternative 1- 9 represents 
different cut-off alternatives for the BCV multiplex (presented in Table 2). The BCV ELISA cut-off was fixed at sample positive > 10 PP for all alternatives. 


Test 


Sub-population 


BCV multiplex BCV ELISA Pop 1 Pop 2 


Parameter 

Cut-off 

alternative 

Se 


Sp 


Se 


Sp 


TP 


TP 


Median 

[95% PCI] 

Median 

[95% PCI] 

Median 

[95% PCI] 

Median 

[95% PCI] 

Median 

[95% PCI] 

Median 

[95% PCI] 

1 

99.9 

[99.4;100] 

77.3 

[69.8;84.8] 

99.0 

[96.9;100] 

99.5 

[97.1;100] 

94.0 

[91.0;96.3] 

61.5 

[56.2;66.7] 

2 

99.6 

[98.6;100] 

91.1 

[85.4;96.0] 

99.4 

[97.8;100] 

97.4 

[93.4;99.7] 

93.5 

[90.5;95.8] 

60.3 

[55.1;65.5] 

3 

99.5 

[98.1;100] 

93.1 

[88.0;97.2] 

99.5 

[98.1;100] 

92.3 

[87.3;96.7] 

93.1 

[90.1;95.5] 

58.0 

[52.7;63.2] 

4 

98.9 

[96.8;100] 

94.3 

[89.5;98.1] 

99.5 

[98.0;100] 

87.5 

[81.1;93.3] 

92.6 

[89.4;95.2] 

55.8 

[50.3;61.2] 

5 

99.0 

[96.8;100] 

94.3 

[89.5;98.1] 

99.5 

[98.0;100] 

81.6 

[75.0;87.8] 

92.2 

[88.9;94.9] 

52.4 

[47.0;58.0] 

6 

97.9 

[95.1;99.7] 

94.3 

[89.5;98.1] 

99.5 

[98.0;100] 

79.8 

[72.6;86.8] 

91.8 

[88.2;94.8] 

51.5 

[45.9;57.3] 

7 

90.6 

[86.6;94.2] 

94.3 

[89.5;98.1] 

99.4 

[97.9;100] 

82.6 

[73.5;92.3] 

92.1 

[88.3;95.0] 

53.3 

[46.8;60.1] 

8 a 

99.9 

[99.3;100] 

93.7 

[88.8;97.8] 

99.5 

[98.1;100] 

99.6 

[97.6;100] 

93.5 

[90.6;95.7] 

61.2 

[56.0;66.2] 

9 a 

97.1 

[94.0;99.4] 

99.2 

[96.9;100] 

99.8 

[99.1;100] 

80.4 

[73.1;87.5] 

91.5 

[87.8;94.5] 

51.6 

[46.1;57.1] 


a Only BCV-A antigen included. 


test, there were no relevant studies we could compare estimates to. 
However, the multiplex technology has been shown useful for bovine 
tuberculosis in cattle and goats (Clegg et al., 2011; O’Brien et al., 2017). 
The parameter estimates provided by the manufacturer for the SVAN- 
OVIR* BCV-Ab are based on data from a study in which 91 serum 
samples were analyzed using both the ELISA and a virus neutralization 
test (VNT) (Alenius et al., 1991). The estimates, Se of 84.6% and Sp of 
100%, were calculated using VNT as gold standard. For the SVANOVIR* 
BRSV-Ab, the Se (94%) and Sp (100%) were calculated in a study 
comparing the test results to another ELISA in 151 serum samples. 
Thus, test estimates were relative to the other ELISA (Elvander et al., 
1995). Results from the former studies are not comparable to the pre¬ 
sent study due to different sample material (serum vs. BTM). Even so, it 
is important to note that in studies assuming a perfect reference test the 
estimated Se and Sp of the index test will never exceed those of the gold 


standard, thus the higher Se of both the BRSV and BCV ELISA found in 
our study was not unexpected. 

To explore the effect of different cut-off values on test character¬ 
istics we applied a range of cut-off values for the multiplex antigens. 
Whenever the cut-off is changed this could entail a change in the de¬ 
finition of the latent condition and change the number of true positive 
and true negative herds. There was relatively little variation in the Se 
and Sp estimates of the BRSV- and BCV ELISA across the different cut¬ 
off values explored, and the change in estimates of true prevalence was 
minor. The tests generally agreed on the proportion of positive herds 
indicating that tests had good agreement on the underlying target 
condition. The explorative approach to choosing cut-offs is a potential 
weakness of the current study; however, the different scenarios provide 
examples of expected performance for different cut-offs and do not 
represent an optimization of the diagnostic tests. The chosen cut-off will 
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likely affect the number of antibody producing animals needed for a 
positive BTM result, and a positive correlation between within-herd 
prevalence and OD-value has been shown for other diseases (Muskens 
et al., 2011; Nekouei et al., 2015). Because the typical Norwegian dairy 
herd is small (mean herd size 25.7) (Anon., 2015) compared to most 
other developed countries, this might influence the generalizability of 
our results: In larger herds antibodies might be diluted in the bulk tank, 
and hence cause the test Se to decrease. However, larger herds might 
also have more positive animals. 

Careful evaluation of the model assumptions is crucial when per¬ 
forming LCA, as violation of assumptions might lead to biased results. 
The assumption of different prevalence between populations is central 
to LCA models, and Toft et al. (2005) showed that the precision of the 
accuracy parameters improved with increasing difference in prevalence 
among the populations studied. In the present study, the difference in 
prevalence between the two sub-populations was relatively large, 
which in addition to a sufficient sample size, leads to narrow posterior 
credibility intervals for the Se and Sp estimates. 

The second assumption is that the test characteristics are constant in 
both populations. The Norwegian dairy herd is relatively homogeneous, 
and the two sub-populations in this study are likely similar in terms of 
breeds and production systems. A potential source of variation in test 
characteristics between sub-populations could be antigenic diversity 
within the Norwegian dairy herd. Findings of antigenic diversity of BCV 
are summarized by Saif (2010) who concludes that only a single ser¬ 
otype is known based on virus cross-neutralization tests, and that a high 
level of cross protection has been shown between respiratory and en¬ 
teric isolates. For BRSV, a Norwegian study found that the current 
Norwegian strains of BRSV belonged to the same subgroup as other 
North European isolates, indicating that the within-country diversity is 
likely to be limited (Klem et al., 2014a). Additionally, cross-reaction is 
likely to be common, and has even been shown for isolates from dif¬ 
ferent species (Oberst et al., 1993). Even though it seems unlikely that 
spatial antigenic diversity plays an important role as source of bias it 
cannot be excluded with complete certainty. 

The final assumption to be met is conditional independence of tests 
given the disease status. Several papers argue that if tests have similar 
biological basis, this assumption is likely not met (Gardner et al., 2000; 
Branscum et al., 2005). Conditional independence between tests means 
that the probability of a positive (or negative) result from one test is the 
same regardless of the result of the other test, given the true disease 
state (Enoe et al., 2000; Toft et al., 2005). Conditional dependence 
would, in terms of false positives, mean that the second test is more 
likely to pick up a herd as a false positive if it already tested (false) 
positive on the first test, for instance due to cross-reactivity with other 
agents. To estimate covariance between tests (ys e and Ysp) two extra 
degrees of freedom are needed. In a two tests, two populations scenario 
this results in an unidentifiable model i.e. it is not possible to estimate 
covariance without including prior information. No reliable prior in¬ 
formation could be obtained for test parameters or prevalences in the 
present study. Another approach potentially allowing for estimation of 
covariance would be to include a third test: either another antibody 

Appendix A. Model description 


test, or a test detecting the virus itself (e.g. a qPCR). The first option 
would not necessarily allow for estimation of covariance unless the 
third test had some underlying properties substantially different from 
the two other tests. Adding an antigen test might ensure conditional 
independence, however, it would change the underlying disease status 
to involve not only serological response, but also a coherent shedding of 
virus. We explored the consequences of conditional dependence (sen¬ 
sitivity analysis) by including fixed covariances as proportions of the 
maximum possible covariance between tests. For the BCV estimates, 
allowing for covariance in the latent class models had negligible effect 
on parameter estimates of both tests. As the Se of the BCV multiplex and 
the Sp of the BCV ELISA is close to one, the small effect of covariance 
was expected. It can be shown mathematically that test Se (Sp) are 
conditionally independent whenever one test has Se (Sp) = 1, see 
Appendix A for details. This was also the situation for BRSV-Se where 
the Se of the ELISA is close to 1. However, the COC-models with fixed 
covariances did yield changes in the estimated specificity for BRSV of 
both tests. This became most notable when the covariance was assumed 
larger than 25% of maximum. In summary, the effect of covariance was 
small except for BRSV-Sp for high values of covariance. It is important 
to note that the sensitivity analysis gives an indication of the effect of 
covariance if present, but does not answer whether covariance exists. 
Even though both tests in this study are antibody tests, they differ in the 
way they are designed. First, the ELISAs uses crude whole virus in the 
ELISA well, whereas the BRSV/BCV multiplex uses peptides and re¬ 
combinant proteins. Second, the tests use different techniques for de¬ 
tection. The ELISAs use a chromogenic substrate and results are based 
on a reading of optical density, whereas the BRSV/BCV multiplex uses a 
chemiluminescent substrate where results are based on a reading of 
light emission. These differences make a violation of the conditional 
independence assumption less likely. 

In conclusion, the BRSV/BCV multiplex and the BRSV/BCV ELISA 
showed similar performance when applied on BTM samples. The Sp of 
the BCV multiplex can be improved by using the BCV-A antigen only, 
and the low Sp of the BRSV ELISA can be improved by increasing the 
cut-off when using this test on BTM. 
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Model description 

The assumption of conditional independence between tests given disease status implies that for the population with infection present (D + ), the 
probability of test 1 and 2 both being positive given the test subject is truly infected is: 

Pr(T 1 + T 2 + |D + ) = Pr(r 1 + |D+)Pr(T 2 + |D+) 

Similarly, for the population of non-infected subjects (D~), the probability of test 1 and 2 both being negative given the test subject is truly non- 
infected: 


Pr(Tfr 2 -|D-) = Pr(Tr|D-)Pr(T 2 -|D-) 
If we define 
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Yse = Pr(T 1 + T 2 + |D+) - Pr(T 1 + |D + )Pr(T 2 + |D+) 
and 

Ys P = Pr(rfT 2 -|D-) - Pr(Tf|D-)Pr(T 2 -|D-), 

of COC between 
POS/POS, POS/ 

0,1 Sej,Spj,Pi ~ Multinominal(Probi, n,) for i = 1,2,...,S and j = 1,2. 


then yse and ysp ate the conditional covariances (COCs) among infected and non-infected test subjects, respectively, and presence 
tests given disease status implies that y Se ^ 0 and/or y Sp ^ 0. 

The latent class model assumes that for the ith subpopulation the counts (O,) of the different combinations of test results, e.g. 
NEG, etc. for the two tests follow a multinomial distribution 


where S is the number of subpopulations; j is the index for the test; and Probi is a vector of probabilities of observing the individual combinations 
of test results for the ith subpopulation (with true prevalence,TP,): 

'Pr(7j + 7’ 2 + ) = Pr(T 1 + T' 2 + |,D + ) + Pr (Teijin-)' 

Prob — Pr(Tl+T2_) = Pl '( T i +T 2 - |£ + ) + PrCr+TTID-) 
r ° ' Pr(TfT 2 + ) = Pr(7’ 1 - 7 2 jD + ) + Pr(Tfr 2 + |ir) 

Pr(rrr 2 -) = Pr(T 1 -r 2 -|D + ) + Vr{T^\D-) t 
(Se 1 Se 2 + y Se )TP f + ((1 - S Pl )(l - Sp 2 ) + y Sp )(l - TP;)' 

(S ei (l-Se 2 ) - r Se )TPj + ((1 - S Pl )Sp 2 - y Sp )(l - TPj) 

((l-S ei )Se 2 - / Se )TP, + (S Pl (l - Sp 2 ) - Xsp)(1 - TPD 
((1-S ei )(l-Se 2 ) + / Se )TPi + (S Pl Sp 2 + r Sp )(l - TP f ) 

The model with CID between tests can be obtained by letting yse = Ysp = 0 in the above expression. 

From the expression for Prob, it is possible to derive upper and lower limits for Yse and Ysp> since each of the elements of the probability vector 
must be between zero and one, thus: 

max[ —(1 - Se 1 )(l - Se 2 ), -Se 2 Se 2 ] < Yse £ minlSejCl - Se 2 ), Se 2 (l - Sej)] 


max[ —(1 - SpiXl - Sp 2 ), -SpjSpJ < Ys P £ minfSpj (1 - Sp 2 ), Sp 2 (1 - Spi)] 


If we let the Se or Sp of either test be equal to 1 in the above equations, it follows that the associated conditional covariance is limited to zero from 
above and below. Thus implying conditional independence (with respect to Se and/or Sp) between the two tests given disease status. In frequentist 
statistics, a 95% confidence interval not including zero is evidence for statistical significance. If a similar approach is adopted in a Bayesian setting, 
then a 95% posterior credibility interval for the conditional dependence without zero indicates that the conditional dependence should be included 
in the model. This covariance can be expressed as either Yse (or Ysp) or as the proportion of covariance relative to its maximum value. 
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